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Abstract 

In this paper, we propose a new noise filtering 
scheme that is based on higher- order statistics (H,0,S,) 
for photographic images corrupted by signal- dependent 
film grain noise. In addition, reliable estimation of 
the noise parameter using H.O,S. is proposed. This 
parameter estimation technique can be used to gener- 
ate film grain noise which has applications in motion 
picture and television productions. Simulation results 
show that the proposed filter perform better than exist- 
ing methods which are based on second- order statistics. 



1. Introduction 

Noise suppression is a common application in image 
processing. Traditionally the noise process is taken to 
be additive white Gaussian and statistically indepen- 
dent of the signal. However, in the case of photographic 
images, it is well known that they contain film grain 
noise that is signal-dependent, that is, the noise statis- 
tics depend on the signal. The noise model describing 
film grain noise and zero mean Gaussian measurement 
noise w has the form [1]: 

r{z, y) =r 8{x, y) + (x, y)n(x, y) + w{x, y) (1) 

where s is the noiseless image measured in density, k is 
the scanning constant, p is an exponent that depends 
on film, n is a Gaussian noise with zero mean and unit 
variance. Because conventional filtering techniques as- 
sume a different noise model, they do not perform well 
in this case. 

Existing methods for suppressing signal-dependent 
film grain noise include modifications of standard tech- 
niques designed for additive noise. Examples are 
the Wiener filter [1] and statistical estimators [2] for 



the above noise model. These techniques yield bet- 
ter performance than those assuming a false signal- 
independent noise. However, the above methods as- 
sume the parameter k is known a prior. Moreover, due 
to the nonlinearlty in the noise model, expressions for 
the statistical estimators (MMSE and MAP) have a 
complicated form even in the special case of Gaussian 
image 5, and involve solving a polynomial equation and 
numerical integration at every pixel. 

In this paper, reliable estimation of noise model pa- 
rameter using higher-order statistics (H.O.S.) is consid- 
ered. In this respect a new filter (Wiener type) based 
on H.O.S. is proposed. Also, realistic film grain noise 
generation, which has applications in television and 
motion picture productions, becomes possible. Because 
measurement noise is Gaussian, higher-order statis- 
tics of the observed image r would contain contri- 
butions from the non-Gaussian image 8 and signal- 
dependent noise only, which leads to better parame- 
ter estimation. Furthermore, since photographic im- 
ages are highly non-Gaussian and film grain noise is 
nonlinearly related to the original image, a lot more 
information can be extracted from their higher-order 
statistics [3]. Filtering schemes based on H.O.S. can 
give better performance. 

2. Design of Higher-Order Statistics 
Based Filter 

Assuming the proposed filter h{x^ y) be a finite im- 
pulse response (FIR) filter with a support region of 

h{x, y) 7^ 0 for a <x <b,c<y <d, (2) 

the filter coefficients /i(x, y) can be solved by minimiz- 
ing a higher-order statistics criterion that is an exten- 
sion of the mean square error (MSE) criterion used in 
the correlation based Wiener filter. Let the error signal 
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e{z,y) be defined as: 

6 d 

y) = s{x, 2/) - X) ^*)''(* - «i y - i)- (3) 

the proposed filter y) is designed by minimizing 
the following criterion [4]: 
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with Cxy = e(a:, y) and h{x^ y) being the optimum filter 
based on the cumulant based criterion Jc - 

3. Cumulant-Based Wiener-Hopf Equa- 
tion 

To compute the optimum filter coefficients, we can 
extend the correlation based Wicner-Hopf equation and 
the orthogonality principle to higher-order statistics. 
By using the idea described in [4], let h{x,y) be the 
filter satisfying the following cumulant based orthogo- 
nality condition: 
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Then it can be shown that h is the optimum filter asso- 
ciated with the criterion Jq . To derive the cumulant 
based Wiener-Hopf equation, we start with the orthog- 
onality condition (Eq. (5)), and substitute expression 
for e(x, y) into Eq. (5) to give: 

6 d 
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where Csr and Crr are defined as 
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By substituting estimates of Crr Csr and forming 
a linear system of equations, filter coefficients /i(t, i) 
can be solved. Note that the above derivation can be 
applied similarly to moments, i.e., moment based crite- 
rion leading to moment based orthogonality condition 
and moment based Wiener-Hopf equation. 

4. Estimation of Higher-Order Statistics 

The use of Eq. (6) requires that Csr and Crr are 
known. In practice, higher-order cumulants are esti- 
mated by replacing the expectation operator by sam- 
ple averaging over the data. Estimating Crr is not 
difficult since we have access to the observed signal. 
However, Csr is not easy to obtain unless we know the 
signal exactly, which is impossible. Thus we need to 
determine their relationships with the original signal 
statistics Csa, which is assumed known. 

By substituting the noise model (Eq. (1)) into the 
definitions of Crr and Csn we have for M = 3 and 
p= 0.5: 



Crrihj) = CsrihJ) + k^{RsiiJ) " mj) 



(9) 



and 
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where Rs and m« are the autocorrelation and the mean 
of the signal 5 respectively: 



Ra{h i) = y)s{x -iyy- j)] 

5. Parameter Estimation 



(11) 



The calculations of Eqs. (9) and (10) require that 
the constant k be known. However, when this infor- 
mation is not available, we must estimate the constant 
from the observed image statistics and the a prior ideal 
image statistics. In the case of p = 0.5, the variance, 
skewness, and kurtosis of the received image are related 
to that of the original image by the following equations: 
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C3 = C3 + SAr^c, 

c4 + 6*^4 - 15k*a^ 



(13) 
(14) 



The value of k can then be solved by substituting the 
statistics of the observed image (which can be esti- 
mated) and the a prior image statistics (<r J , cj, and 
cj) into any of the above equations. Note from the 
above equations that the use of higher-order statistics 
in the presence of Gaussian measurement noise leads to 
better estimation of Jb, as cumulants of Gaussian noise 
are identically zero. 

6. Film Grain Noise Generation 



Figure 1. Test image: 'Mountain'. 




As outlined in Introduction, film grain noise gener- 
ation has applications in television and motion picture 
productions since digitized film images, video images 
and computer generated images are routinely combined 
into one frame. In this process artificial noise is added 
to the video and computer generated images to match 
the grain pattern of the film. To generate the right 
amount of artificial film grain noise, the noise parame- 
ter k must be known. 

Using the above method to estimate the noise par 
rameter k would require statistics of the corrupted and 
ideal images. When k is solved, generate and add a 
noise image to the computer generated image accord- 
ing to E)q. (1), but without the measurement noise w. 
In cases where the a prior ideal image statistics are 
not known, an approximate solution was proposed in 
[5] which uses statistics of a filtered image as the ideal 
image statistics. The observed image r is processed us- 
ing a sub-optimal filtering technique that requires no 
knowledge of the noise statistics. 

7. Simulation Results 

In this section, the above proposed methods in esti- 
mation of the parameter k^ and signal-dependent film 
grain noise removal based on H.O.S. are applied. Two 
test images of size 256 x 256 were used: Lenna and 
Mountain. Test image 'Mountain' is shown in Fig. 1. 

To test the validity of parameter estimation of A:, a 
number of simulations were performed for the following 
two cases: 

• signal-dependent noise only 

• mixture of signal-dependent / signal-independent 
noise 

Signal-dependent film grain noise and Gaussian mea- 
surement noise are added to the image 'Lenna'. Sample 



r r 



cumulants are calculated using the following relation- 
ships: 

(15) 
(16) 
(17) 
(18) 



C3 = 17*3 — 3m|m2 + 2(mj)^ 
cl = mJ-4mIm5-3(m$)Vl2(mi)2m5-6(m5)^ 

The quantities m^, 9712, 7713, and are estimated from 
an M X iV image using sample averaging. The signal 
statistics <tJ, cj, and C4 of the image *Lenna' are known 
a prior and are used to solve for k. The parameter p 
was fixed to be 0.5 throughout the experiments since 
this is typical for a variety of film stocks. Because the 
variance of the signal-independent noise is assumed un- 
known, k is solved using Eq. (12) with zero measure- 
ment noise variance. A value of k = 0.1 was selected 
for moderate noise corruption. The value of k deter- 
mines the degree of degradation, as can be seen from 
the variance of the signal-dependent noise term: 



noise 



(19) 



Fifty independent runs were performed, with the re- 
sults summarized below. 





estimated k (meand: standard deviation): 




2nd order 


3rd order 


4th order 


0 


0.0999^0.0015 


0.0999±0.0024 


0.0998±0.0033 


0.05 


0-1299±0.0014 


0.1003db0.0030 


0.1006db0.0041 


0.10 


0.1939±0.0013 


0.1013±0.0039 


0.1009±0.0072 


0.15 


0.2681db0.0015 


0.1004±0.0058 


0.1007^0.0082 


0.20 


0.3465±0.0014 


0.0990±0.0096 


0.0984±0.0159 



The advantage of using H.O.S. in estimation is evi- 
dent from the tables. Second-order statistics results in 
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high bias and low variance, whereas estimation using 
H.O.S. has low bias and higher variance. It is clear that 
estimating k using H.O.S. is better than using second- 
order statistics in the presence of Gaussian measure- 
ment noise. 

For noise filtering, again two different cases were in- 
vestigated: 

• signal- dependent noise only 

• mixture of signal-dependent/signal- independent 
noise 

A value of /r = 0.1 was used. For a mixture of noise, 
variance of measurement noise cr^ was chosen to be 
0.005. 

The criteria used in evaluating the performance were 
1) signal-to-noise ratio (SNR), 2) mean absolute error 
(MAE), and 3) mean square error (MSE) which is sim- 
ilar to SNR. These are defined below for an image of 
size M X N: 



SNR = 10 logio 



(20) 



M-lN-1 



MAE = — 53 53 [s(x, y) - s{x, y)] (21) 
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(22) 
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where s and s are the ideal and estimated images, re- 
spectively. A filter size of 3 x 3 was chosen. Wiener fil- 
ter designed based on the usual correlation based crite- 
rion [1] (designated M2) and the two proposed higher- 
order statistics based criteria (third-order moment M3 
and third-order cumulant C3) are compared. 
A number of observations can be made: 

• On average higher-order statistics based filters 
achieved better SNR than the correlation based 
filter by 1 dB. This improvement may be due to 
the fact that more information about the image 
statistics was utilized. 

• The cumulant based filter performed as good as 
the higher-order moment and correlation based fil- 
ters in the case of signal-dependent noise only. 
However, for a mixture of noise, the cumulant 
based filter is not suitable. By examining the cu- 
mulant based Wiener-Hopf equation in Eq. (6): 




HhJ)Crr{p-'i.g-j) (23) 



we have for a mixture of noise: 

C,r ( mixture of noise ) 
« Csr ( film grain noise only ) 



(24) 



and 



Crr ( mixture of noise ) 
« Crr ( film grain noise only ) 



(25) 



thus the cumulant based criterion cannot not rec- 
ognize the Gaussian measurement noise. The filter 
performed as if only film grain noise is present. 

• Performance of cumulant based filter depends 
heavily on the properties of the image. For third- 
, order cumulant, if the distribution of an image is 
close to Gaussian or is symmetric, then it is not 
appropriate to use the cumulant based filter. Ta- 
ble 5 shows the statistics of the two images. It 
can be observed that 'Mountain' has the lowest 
skewness, thus third- order cumulant based filter 
did not perform well. If fourth-order cumulant is 
used, the filter should perform well in the case of 
film grain noise only, as indicated in Table 4. 

To test the noise generation procedure, the image 
'Lenna' was used for noise generation. Corrupted im- 
age was filtered using the method described in [5]. 
Then k was computed using fourth-order statistics of 
the two images. Although k can be solved by match- 
ing their variances, it was found that variance of the 
filtered image is lower than that of the corrupted image 
because edges are blurred to some extent. Thus using 
variance to obtain k would lead to over-estimation, and 
the final image would be too noisy. To compare the 
noise level of the original corrupted and the final im- 
age, SNR was used. For the original corrupted image 
the signal power is the ideal signal power, whereas the 
signal power of the final image is that of the filtered 
image. It can be seen from Table 6 that the noise level 
in two images (noise-added and original corrupted im- 
ages) are about the same. 

8. Conclusions 

This paper presents a H.O.S. based filter for filtering 
images corrupted by signal-dependent film grain noise. 
In addition, estimation of noise parameter using H.O.S. 
is proposed and successfully applied in film grain noise 
generation. Simulation results show that the perfor- 
mance of this filter is better than that of filter based 
on second-order statistics, and parameter estimation 
using H.O.S. is more reliable in the presence of Gaus- 
sian measurement noise. 
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Table 2. Test image 'Lenna' with signal-dependent noise 
only (k=0.1). 





SNR (dB) 


MAE 


MSE 


unfiltered 


16.7591 


4.6375e-2 


3.6820e-3 


Afa 


19.8458 


3.1849e-2 


1.8089e-3 


Ma 


19.9902 


3.1212e-2 


1.7498e-3 


Cz 


20.1143 


3.0150e-2 


1.7005e-3 



Table 3. Test image Xenna' with mixture of 
signal-dependent noise (k=0.1) and measurement noise 



= 0,005). 





SNR (dB) 


MAE 


MSE 


unfiltered 


13.0270 


7.3938e-2 


8.6957e-3 


Ma 


17.7796 


4.1604e-2 


2.9110e-3 


Ms 


17.8464 


4.1266e-2 


2.8666e-3 


Ca 


13.7675 


6.7819e-2 


7.3494e-3 



Table 4. Test image 'Mountain' with signal-dependent 

noise only (k=0.1). 





SNR (dB) 


MAE 


MSE 


unfiltered 


15.9772 


4.52806-2 


3.40056-3 


M2 


20.9042 


2.5644e-2 


1.0936e-3 


Ms 


21.6888 


2.1905e-2 


9.1283e-4 


M4 


21.5968 


2.2600e-2 


9.32376-4 


a 


20.7989 


2.3356e-2 


1.1204e-3 


C4 


21.7648 


2.2393e-2 


8.9398e-4 



Figure 2. Corrupted 'Mountain' with = 0.1. 




Figure 3. Filtered 'Mountain* using M2 criterion- 




Table 5. Image statistics of various test images. 





Image Statistics 


Image 


mean . 


variance 


skewness 


Lenna 


3.6363e-l 


4.2353e-2 


7.2266e-3 


Mountain 


3.3807e-l 


2.0373e-2 


1.0025e-3 



Table 6. SNR of noise-generated images using different 
image statistics (original SNR is 16-7620 dB), 



statistics 


SNR (dB) 


second order 


16.0929 


third order 


15.8685 


fourth order 


16.9721 



Figure 4. Filtered ^Mountain* using C4 criterion. 
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